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Abstract 

The Otherside is an advanced open-source multime¬ 
dia system, designed to run as a server application. 
It is essentially a sound synthesis system that al¬ 
lows for the creative collaboration of a number of 
people or machines over computer networks. Un¬ 
like most similar attempts to create network-based 
audio applications, the Otherside does not require 
any specialized audio software or massive amounts 
of computer processing power. The audio processing 
is done on the server side and directed to listeners 
using a simple Internet Radio protocol. Any listener 
can then choose to participate in the sound creation 
process by using a simple web-browser-based chat¬ 
room, or by utilising their own advanced systems for 
network control. 
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1 Introduction 

1.1 Creative Collaboration 

The Otherside Server provides a platform for 
the creative collaboration of any number of peo¬ 
ple, regardless of their understanding of mu¬ 
sic, computer programming or networking. It 
is platform-independent, meaning that it will 
run properly on any operating system and any 
computer architecture. This makes it easily ac¬ 
cessible to a large number of people who can 
work together and even communicate, during 
the course of their session with Otherside. 

People can create music together, from ran¬ 
domly changing values and creating random 
sonic events to carefully crafting their own soft¬ 
ware that will interface with the Otherside like 
an additional musical instrument. It is an en¬ 
trance to the world of computer music for peo¬ 
ple who have no such previous experience. It 
can also serve as an advanced experimentation 
platform for users who are already familiar with 
the protocols and principles involved. It is open 
and accessible to several different protocols and 
adheres to the open-source mentality, meaning 


that users can get involved to the extent of be¬ 
coming developers of the Otherside system. 


1.1.1 Computer Network Collaboration 

Computer Networks are nowadays dominating 
the world through the internet. However, most 
of the well-established network applications are 
very simple and are not particularly demand¬ 
ing in processing power and network bandwidth. 
Digital Signal Processing is a complicated task 
that only recently became a “household” appli¬ 
cation on personal computers. Careful planning 
and consideration is required to overcome some 
of the difficulties posed by computer networks 
and their use by artistic collaboration systems. 

One of these problems is that network con¬ 
nections and especially the internet often in¬ 
volve extremely long distances between two 
users, thus inevitably data can not be trans¬ 
ferred seamlessly. There is an evident latency 
issue that is being treated differently by such 
project developers. Jeorg Stelkens discusses this 
in a paper about his own network collabora¬ 
tion systemfl], proposing the integration of la¬ 
tency in the creative process, as a unique char¬ 
acteristic of the computer network as a medium. 
The Otherside does not attempt to overcome or 
to cover-up latency, unlike most projects men¬ 
tioned by Stelkens. It merely accepts latency as 
a fact and uses the amount of users simultane¬ 
ously transmitting data to create a sonic situa¬ 
tion where events can either be left to chance, 
creating aleatory music [2], or forcing a user to 
take the amount of latency into consideration 
when crafting a sonic event in order to be able 
to predict the outcome. This approach is based 
on the principle of integrating the latency in the 
creative process and passing on its acceptance 
as a fact to the users, who will have to take it 
into account when using this system. 



1.2 Related Creative Development 

1.2.1 NetPD 

An inspiring project, leading to the ini¬ 
tial thoughts for creating the Otherside was 
NetPD [3]. NetPD is based on Pure-Data[4] and 
enables a group of people to collaborate over a 
computer network. The idea of collaboration 
is common between NetPD and the Otherside, 
but the Otherside attempts to break the limit 
of keeping it only between users of Pure-Data, 
who would have to be comfortable with data¬ 
flow programming and networking. The Oth¬ 
erside allows users to become involved as much 
as they please. NetPD also features a chat fa¬ 
cility within Pure-Data, for communication be¬ 
tween participants. Otherside is based on IRC, 
a well-known chat interface, but goes much fur¬ 
ther than just using it for communication. It 
is worth noting that it is technically possible to 
connect NetPD and the Otherside and create a 
collaboration between the two projects. 

1.2.2 Pure-Data and Data-Flow 
Programming 

Pure-Data was created by Miller Puckette, as 
an open-source project, similar to his Max[5] 
program. It is a data-flow programming lan¬ 
guage geared towards audio/visual output and 
control. Pure-Data is a direct descendant of 
Max, written in 1988 [6] by Miller Puckette while 
he was in the Institut de Recherche et Co¬ 
ordination Acoustique/Musique, Paris, France. 
David Zicarelli together with Puckette later 
wrote MSP as an addition to Max, thus creat¬ 
ing Max/MSP, enabling audio-rate digital sin- 
gal processing. Max was only capable of control 
rate processing, as computers in 1988 were not 
fast enough to handle audio-rate signal process¬ 
ing. Nowadays data-flow programming is used 
widely for audio applications, as it is a very 
powerful and versatile way of interfacing with 
various control protocols. Computers nowadays 
have enough processing power to support heavy 
digital signal processing applications and net¬ 
working speeds are high enough to be used cre¬ 
atively in interactive applications. This makes 
data-flow programming languages an interest¬ 
ing option for users who want to explore the 
field of complex control in sound generation sys¬ 
tems. 


2 Technical Description 

2.1 Operating System 

The Otherside is based on open-source prin¬ 
ciples, being built on the Ubuntu Linux 8.04 
Server Edition operating system. The operat¬ 
ing system was chosen as it has proved to be 
very stable as a server system while being di¬ 
rectly compatible with all the audio software 
and techniques that were to be used. It is also 
an operating system that is very open to cus¬ 
tomization, something that was necessary to en¬ 
sure high performance under the load of audio 
processing and server applications. 

The preparation of the computer system on 
which the Otherside was tested started with the 
installation of the base Ubuntu 8.04 Server sys¬ 
tem, replacing the Linux Server Kernel with the 
Linux 2.6.24 RT 1 Kernel, which allows audio 
processing to be done with Real-Time priority. 
Since audio processing is one of the main func¬ 
tions of the Otherside, real-time implementa¬ 
tion was an important step towards high per¬ 
formance. To enable the audio to function on 
the Ubuntu Server operating system the instal¬ 
lation of ALSA[7] was also required. 

2.2 Web interface 

To achieve an extreme level of accessibility by 
users who have no interest in turning their per¬ 
sonal computer into an audio workstation, all 
functions of the Otherside are accessible from a 
simple website interface. 

The starting point is a website that is pow¬ 
ered by the Apache 2 HTTP server [8], the most 
widely used HTTP server on the internet. This 
links to a Streaming Audio Server, an IRC Chat 
Interface, a help document simply explaining 
the basics of the Otherside to the users and an 
email link for communication with the adminis¬ 
trator of the system. Apache 2 was chosen due 
to the fact that all other HTTP servers under 
investigation 2 were simply not as advanced and 
reliable, something which is also evident by the 
widespread use of the Apache on the internet. 

The Streaming Audio Server is using the Ice- 
cast 2 Server [9] to provide a compressed audio 
stream produced by the Otherside to an au¬ 
dience with an internet connection and a me¬ 
dia player on their computer. The Icecast 2 

1 An alternative Kernel, with Real-Time capabilities 

2 Examples of other HTTP Servers that were consid¬ 
ered instead of Apache include Mathopd, micro-httpd, 
mini-httpd and nanoweb 



Server receives a compressed audio stream, util¬ 
ising MPEG Layer 3 compression, forwarding 
it to a number of listeners over a computer 
network. This is based on the established In¬ 
ternet Radio Station practice of using an en¬ 
coder that streams audio to a Streaming Audio 
Server to be forwarded to any number of listen¬ 
ers. The difference here is that instead of encod¬ 
ing sound files, the encoder is directly encoding 
and streaming the computer-generated sound of 
the synthesis engine that powers the Otherside. 
The link from the main website redirects a user 
to the website interface of the Icecast 2 Server, 
which contains details about the audio stream 
and a link for listening. Icecast 2 was chosen be¬ 
cause it is a very well-made open-source Stream¬ 
ing Server with a friendly user interface. 

The control interface was a challenge as there 
were too many functions that needed to be im¬ 
plemented in a simple and accessible interface. 
The choice of an IRC Chat interface as the con¬ 
trol interface was made as the nature of IRC 
already implemented a lot of the functions that 
were needed, without the need of reinventing 
the wheel. Several different alternatives for the 
IRC Server software were tried, but most of the 
common servers 3 proved to be unsuitable for 
the kind of use that was about to be adopted. 
The RFC1459[10] and RFC2813[11] specifica¬ 
tions include several functions geared towards 
huge IRC Server networks, which were a burden 
on CPU power and configuration time, there¬ 
fore clearly not wanted in the Otherside. The 
server software which was finally used was the 
IRC-Net IRC 2 Server, as it was very simple 
to configure, reliable and somewhat modular in 
design, allowing the administrator to engage or 
disengage any modules needed for the task. By 
default, it comes with the bare minimum nec¬ 
essary to function, which is exactly what the 
Otherside requires. 

Usually, an IRC Network has ways of regis¬ 
tering nick-nanres and chatrooms. This is to 
allow frequent users to always use their own 
nickname while prohibiting anybody else from 
using it, while there is also a service for leav¬ 
ing messages to registered users that are cur¬ 
rently offline. These are collectively known as 
IRC Services. The decision against using any 
IRC Services Provider package came naturally 
at this stage, as the IRC Server used with the 

3 Servers that were considered include the Undernet 
IRC Daemon, the Ratbox IRC Daemon and the Hybrid 
IRC Daemon 


Otherside is merely a control interface, there¬ 
fore there is no need for registering a nick-nanre 
or a chat-room. 

An IRC Server also has the ability to connect 
to other IRC Servers, forming IRC Networks. 
This was one of the most impressive features of 
the IRC protocol, as it means that there is room 
for a great amount of evolution for the Oth¬ 
erside, by connecting to other Othersides and 
forming a huge Otherside Network in a very sim¬ 
ple and efficient way. This is also an easy way to 
get extensive load off of one server’s central pro¬ 
cessing unit, by interfacing several computers to 
run the Otherside System and sharing the load 
of processing in a clustered-computing fashion. 

However, an IRC Server is only the back-end, 
requiring a Client program 4 for user interaction. 
Most Client programs assume a certain level of 
understanding from the user about the IRC Pro¬ 
tocol. Investigation on this matter led to the 
use of CGLIRC, a very smart way of creating a 
web-browser interface for the IRC Server, thus 
eliminating the requirement for an IRC Client 
program to be installed on the client computer. 
CGLIRC runs on the server machine and cre¬ 
ates a CGI-based web-page that can run on the 
majority of web-browsers, allowing the user to 
connect to and use the IRC Server from this 
web-based client. 

This IRC interface allows a number of users 
to connect to a chat-room and type in mes¬ 
sages. The interaction between the users and 
the server was achieved by using an IRC In- 
fobot, a computer program that sits on an IRC 
Server, posing as a human user, that replies 
to certain messages with predefined actions. 
These messages usually contain information re¬ 
garding the Server. This idea let to the cre¬ 
ation of an Infobot-like program that would 
listen for specific messages, but instead of re¬ 
plying back to the user that sent the mes¬ 
sage, it would respond by directing them to 
the Sound Synthesis Engine. After some in¬ 
vestigation, it became evident that there was 
nothing similar readily available. Thus came 
the decision to create an original control-bot us¬ 
ing a computer programming language. Con¬ 
sideration of C/CH—h[12] and Python[13] led 
to a decision against C/C++ as it was lack¬ 
ing some of the easy-to-use functions of Python 
for string formatting. By studying some exist- 

4 Common IRC client programs include mIRC for the 
Windows operating system and bitchX for POSIX sys¬ 
tems 



ing Infobots such as the XoR bot skeleton[14] 
and the Redland Bot [15] it became clear how 
they worked and ideas were implemented in the 
new code. The source code was loosely based 
on the XoR bot skeleton as it had some inter¬ 
esting string formatting examples. The Sim- 
pleOSC Library[16] for the Python program¬ 
ming language enabled the reformatting and 
redirection of messages to the Sound Synthe¬ 
sis Engine, using the extremely versatile OSC 
protoco[17]. The program also uses the Sockets 
Library for TCP/UDP network connections to 
connect to the IRC Server and to redirect all 
undefined messages sent by users to a module 
on the Sound Synthesis Engine that treats it as 
raw American Standard Code for Information 
Interchange data. Finally, some of the existing 
Infobot funtions of the XoRbot were retained al¬ 
most intact, especially regarding the IRC Proto¬ 
col channel functions and also to include a short 
help message and the basis for future Musical 
Instrument Digital Interface implementation. 

The users that log on to the chat-room see 
a user with the nick-nanre “Otherbot”, which 
is the bot that interfaces the IRC Server with 
the Sound Synthesis Engine. The only thing 
they need to do to control the Sound Synthe¬ 
sis Engine is send messages to the bot as if 
they were chatting to a friend, from their web- 
browser window. 

2.3 Sound Synthesis Engine 

The Sound Synthesis Engine is based on a ver¬ 
sion of the Pure-Data data-flow programming 
software, called PD-extended. It is modular 
in design and consists of a Polyphonic Addi¬ 
tive Synthesis[18] Module, a Wavetable Syn¬ 
thesis [19] Module, a Single Sideband Modula¬ 
tion [20] Module, a Ring Modulation[21] Mod¬ 
ule, a Comb Filter[22] Module, a Stereo Delay 
Line [23] Module and a Speech Synthesis Mod¬ 
ule. The Speech Synthesis Module is not based 
on PD-extended. Instead, it is based on the Fes¬ 
tival Speech Synthesis System[24], developed by 
the Centre for Speech Technology Research of 
the University of Edinburgh, UK. The partic¬ 
ular design of the sound synthesis engine was 
geared more towards the demonstration of the 
capabilities of the control interface, rather than 
a complete creative system. There is much more 
that can be done with the synthesis system and 
the administration setup of the Otherside makes 
it easy to update the existing patches or upload 
new ones, adding to the system. 


Figure 1: Block Diagram 



The Otherbot outputs three kinds of mes¬ 
sages towards the Speech Synthesis Engine. The 
easiest to explain is probably the Raw ASCII 
Data. When a user types a message in the 
chatroom that is not recognised as a predefined 
command, the bot establishes a User Datagram 
Protocol connection to port 9998 of the server 
and redirects the full message. UDP Port 9998 
is opened by a PD-extended object listening for 
inbound connections, thus acting as a daemon 
program. The data is received as ASCII inte¬ 
ger values. These are treated as MIDI pitch 
values and are directed to a polyphonic addi¬ 
tive synthesis module, which creates the equiv¬ 
alent of each MIDI message received, spatially 
panning it according to the number of char¬ 
acters received in a message. The odd num¬ 
bers are panned to the left while the even ones 
are panned to the right. Therefore a message 
length of three characters would have the first 
and third characters panned to the left while the 
second character is panned to the right, after be¬ 
ing converted from characters to ASCII values 
representing MIDI pitch values and then synthe- 










sized to sound. The sound is then directed to 
a Stereo Delay Module for further processing. 
The spatialization in this module is evidently 
very simple, something which will be rectified 
in future versions, by replacing the current algo¬ 
rithm with one that does proportional panning. 
It currently acts as a mere demonstration of the 
potential of the Otherside. 

When a message typed in the chatroom 
matches a predefined command recognised by 
the Otherbot, this is redirected as OSC data 
by establishing a UDP connection to port 9997 
of the server. UDP Port 9997 is opened by a 
PD-extended object designed to receive OSC 
data and direct it to the equivalent OSC path. 
These OSC paths direct data to the appropriate 
objects in the PD-extended patch, where they 
control certain functions, such as the speed at 
which the wavetable is read or the delay time 
and feedback. 

The Wavetable Synthesis Module consists of 
two saved wavetables. One is used to define 
a waveform while the other is used to define 
the speed at which the waveform is being re¬ 
produced. When the speed is altered using the 
appropriate OSC command, the waveforms are 
being read at different speeds, so the whole pro¬ 
cess can be sped up or slowed down according 
to taste. 

The outputs of the Wavetable Synthesis Mod¬ 
ule are directed to two Single Sideband Modu¬ 
lators. These are used primarily for frequency 
shifting and general “warbling” effects. The 
output is then directed to the Stereo Delay 
Module. 

The Speech Synthesis module is a bit more 
complicated since it does not solely rely on PD- 
extended to complete its task. If a message in 
the chatroom is preceded by the “talk” com¬ 
mand, then anything following the command is 
being written in a buffer text file. Once this 
is done, the bot calls a system function that 
runs Text2Wave, a program that comes with 
the Festival Speech Synthesis System that con¬ 
verts text files to audio files. The command tells 
Text2Wave to use the buffer text file as an in¬ 
put file and synthesize the contents of this file 
to sound, saving it as a PCM Wave file. The 
PD-extended module that deals with the Speech 
Synthesis implementation looks for this file and 
plays it in a loop until the buffer text file is 
altered. When this happens, the PCM Wave 
file is altered as well, so the next time the loop 
is started it starts playing the new file. The 


Speech Synthesis module sound output is then 
fed to a Ring Modulator. The Ring Modula¬ 
tor module multiplies the sound with a carrier 
wave. The frequency of the carrier wave is con¬ 
trollable by OSC, enabling a user to actively 
control how the voice is effected during play¬ 
back. 

The output of the Ring Modulator module 
is directed to the Comb Filter module. This 
creates four instances of the input signal and 
combines them, applying slight delays to each 
instance. The level of each of the four instances 
is changed sequentially according to a time vari¬ 
able. The time variable can be altered via OSC 
as well. The output of the filter is directed to 
the Stereo Delay Module. 

The Stereo Delay Module is a delay line with 
a feedback loop. It was based on an earlier de¬ 
sign of the author that also had a Graphical 
User Interface. This was used on the “Weird 
Synthesizer for Weird People” [25] project and 
was based on a design by Jason Plumb[26]. 

2.4 Encoding 

The master signal output of the PD-extended 
patch is not directed to the Digital/Analog Con¬ 
verter object as is most often the case, since we 
are not interested in directing the sound to the 
sound-card of the computer it runs on. Instead, 
it is sent directly to the encoder, an “mp3cast” 
object that is part of the “Unauthorized” li¬ 
brary [27], written by Yves Degoyon. This en¬ 
codes the sound to the desired format. In this 
case, it uses the “LAME” library[28] to encode 
the audio into an MPEG Layer 3 stream and 
send it to the Icecast 2 Server, directly from 
PD-extended. 

Several different ways of doing the conversion 
and streaming to the server had been investi¬ 
gated, prior to deciding on the final solution. 
Methods that were investigated included com¬ 
piling Darkice' 5 with LAME library support to 
enable it to encode using the MPEG Layer 3 for¬ 
mat. However, Darkice required an input by a 
program that was compatible with it. An early 
experiment was to use a “DAC” object in PD- 
extended, together with Jack 6 , a signal routing 
program for ALSA. Darkice had to be recom¬ 
piled with support for Jack. The output of the 
“DAC” object was then disconnected from the 
sound-card input and was directed to the in¬ 
put of Darkice instead. However, this set-up re- 

5 An audio streamer, written by Akos Maroy 

6 Jack is a virtual patchbay for signal routing 



quired two additional programs to be compiled, 
installed and running on the server, Darkice and 
Jack Daemon. Although this might have been 
more efficient used with certain computer sys¬ 
tems, in this case it proved to be a burden on 
CPU power. 

2.5 Creating the Bridge 

So far, it has been made clear that the Other- 
side depends on several existing programs, with 
a bridging system that makes it all work to¬ 
gether towards the desired result. The IRC in¬ 
terface acts as the main bridging point. The 
users send messages to a bot and this bot sends 
commands to the rest of the programs. This 
works by listening for specific key phrases. If 
it detects any of the key phrases in the be¬ 
ginning of a message sent by a user, it inter¬ 
prets the text that follows in the appropriate 
manner. These key phrases are help, osc, talk, 
and midi. If the user types “help” in the cha¬ 
troom or in a private message to the bot, they 
will get a help message in response. If they 
type “midi”, they will get an information mes¬ 
sage in response, letting them know that midi 
has not been implemented yet. “Talk” sends 
the text following the command to the Festi¬ 
val Speech Synthesis System that converts this 
into a Wave file. OSC control is slightly more 
complicated. The user needs to type in the ar¬ 
guments needed by the receiving end, which is 
PD. The message starts with “osc” as in the 
previous examples, this phrase is followed by 
the OSC path, for example /reverb/time, and 
the values that we want to pass to this path, 
for example 800. The whole message would be 
osc /reverb/time 800. This tells the bot to 
interpret /reverb/time 800 as an OSC mes¬ 
sage, forwarding it to PD. PD receives this mes¬ 
sage on UDP port 9997, and redirects the value, 
800, to the appropriate path. In effect, this sets 
the reverb time to 0,8 seconds. 

This way of parsing OSC messages might at 
first seem counter-intuitive. However, consider¬ 
ing that we are doing this over IRC, a few tra¬ 
ditional possibilites immediately come to mind. 
For instance, a user can set up their own IRC 
bot, connect it to that chatroom and instruct 
it to receive MIDI messages from a MIDI con¬ 
troller connected to the user’s computer, con¬ 
vert them to OSC, and send them through a pri¬ 
vate chat to the OtherBot in the manner spec¬ 
ified above. This would create a MIDI-to-OSC 
bridge, over a network. While manually typing 


Figure 2: General Block Diagram 



an OSC message is probably not of much use in 
itself, it can open up amazing possibilities for 
automated control ideas to be implemented. 

On the other hand, there will probably be 
a fair amount of human communication taking 
place in any IRC chatroom. This data can be 
used as a kind of control in itself. All messages 
that the bot receives, but cannot interpret as 
any known message type, are stripped of the 
IRC commands that come with them and are 
sent to PD as raw text messages. PD converts 
it to ASCII data and uses it. So even when the 
bot receives no evidently useful data, it still “re¬ 
cycles” seemingly random data to keep things in 
motion. 

3 Conclusion 

The Otherside is unique in several ways, com¬ 
pared to other examples of network collabora¬ 
tion systems. First and foremost, from a tech¬ 
nical point of view, all processing is done on the 
server side and the sound is transfered to the lis¬ 
teners and users as an audio stream. This means 
that all users are listening to the same sonic out- 









put. A common problem associated with com¬ 
puter network collaboration systems is the diffi¬ 
culty is synchronisation of the audio due to net¬ 
work latency, especially in a situation where the 
collaboration might be between several users in 
the same room. If each of the users is producing 
audio independently, it is technically impossi¬ 
ble to ever synchronize the output. However, in 
the Otherside system, since all users are work¬ 
ing towards a common sonic piece that is being 
created on the server side, only one computer 
needs to act as the audio reproduction client in 
each such setting. Therefore, in the collabora¬ 
tion situation where users are in the same room, 
the audio only needs to be streamed to one com¬ 
puter, coming out of one pair of loudspeakers. 
There are no synchronization problems, since all 
the users are doing is transmitting control data 
to the server, which creates one sonic artwork 
by synthesizing transmitted data in the order of 
arrival. 

In the case of a network of Othersides, the 
servers all share the same user control data as 
IRC servers share all data between them when 
networked. However, each Otherside Server 
runs its own sound synthesis engine so audio 
data does not need to be transmitted between 
servers. All servers will create the exact same 
version of the audio stream, since they have re¬ 
ceived the same control data, and stream it in¬ 
dependently. This is extremely useful as users 
can stream the audio off their local server, while 
being able to perceive the input of a user con¬ 
nected to a different Otherside Server. This 
function makes the possibility for a true global 
collaboration very realistic. 

The use of IRC as the main bonding point 
that controls everything allows for simple and 
effective automation methods to be used. Code 
from IRC bots and utilities can be re-used and 
altered, thus making it very easy for a program¬ 
mer to get started in creating a control interface 
to work with the Otherside. 

There is also an evident social side to the Oth¬ 
erside system, since the use of an IRC proto¬ 
col allows users to easily communicate as they 
would in normal chat-rooms. They are given 
the opportunity to start their own chat-rooms 
and form “teams” of users that cooperate for a 
desired sonic experiment. This can easily grow 
into a network of Otherside Servers in different 
parts of the world, who could collaborate in cre¬ 
ating sound or developing the Otherside system. 
The IRC protocol is currently not being used at 


its full potential for chat services. It can easily 
evolve into a full IRC chat interface, with ser¬ 
vices enabling users to have a personalized iden¬ 
tifier/nickname and build permanent subject- 
specific chatrooms. The Otherside presents the 
world with the opportunity to turn the idea of 
computer network collaboration platforms into 
a global social network. 

Location 

The Otherside server can be ac¬ 
cessed during the Linux Audio Con¬ 
ference 2009 on the following URL: 
http://otherside.servebeer.com/otherside 
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